Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python): Enable zstd compression by default #1384

Merged
merged 25 commits into from
Jan 14, 2025

Conversation

angus-langchain
Copy link
Contributor

No description provided.

@angus-langchain angus-langchain force-pushed the angus/enable-compression branch from 3300712 to ec73127 Compare January 7, 2025 19:10
@angus-langchain angus-langchain changed the title Angus/enable compression feat(python): Enable zstd compression by default Jan 8, 2025
python/langsmith/_internal/_background_thread.py Outdated Show resolved Hide resolved
python/langsmith/_internal/_background_thread.py Outdated Show resolved Hide resolved
threading.Thread(
target=tracing_control_thread_func_compress_parallel,
args=(weakref.ref(client),),
).start()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should return after here right? Otherwise we would hit the main loop for this thread, which we don't need if we're using the run compression strategy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the thread needs to stay active at least until we can drain all the jobs from the tracing queue before we switch to compression. Thinking about the best way to do this

@angus-langchain angus-langchain marked this pull request as ready for review January 9, 2025 01:19
python/langsmith/_internal/_background_thread.py Outdated Show resolved Hide resolved
python/langsmith/client.py Outdated Show resolved Hide resolved
python/langsmith/_internal/_background_thread.py Outdated Show resolved Hide resolved
python/langsmith/_internal/_background_thread.py Outdated Show resolved Hide resolved


@patch("langsmith.client.requests.Session")
def test_create_run_with_zstd_compression(mock_session_cls: mock.Mock) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would also add a test to ensure that the case where the info endpoint does not support zstd_compression_enabled uses regular multipart

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additoinally, need a test case for DISABLE_RUN_COMPRESSION

python/langsmith/utils.py Show resolved Hide resolved
@angus-langchain angus-langchain changed the base branch from main to py-version-0.3.0 January 13, 2025 21:35
@angus-langchain angus-langchain merged commit bb1c05c into py-version-0.3.0 Jan 14, 2025
5 checks passed
@angus-langchain angus-langchain deleted the angus/enable-compression branch January 14, 2025 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants